Method For Controlling A Robot And/Or An Autonomous Driverless Transport System

ABSTRACT

A method for controlling a robot and/or an autonomous driverless transport system on the basis of a sensor-based identification of objects includes generating. Point pair features of the 2D surface contours on the basis of 2D surface contours of the objects to be identified. A point cloud of the environment is acquired using a distance sensor, a surface normal is estimated for each point, and corresponding point pair features of the environment are generated. In a voting method, environment features are compared with model features to efficiently generate pose hypotheses, which are subjected to an optimization and a consistency check in order to ultimately be accepted or rejected as an acquisition.

CROSS-REFERENCE

This application is a national phase application under 35 U.S.C. § 371of International Patent Application No. PCT/EP2016/001174, filed Jul. 8,2016 (pending), which claims the benefit of German Patent ApplicationNo. DE 10 2015 212 932.2 filed Jul. 10, 2015, the disclosures of whichare incorporated by reference herein in their entirety.

TECHNICAL FIELD

The present invention relates to a method for controlling a robot and/oran autonomous driverless transport system, and a corresponding transportsystem.

BACKGROUND

A large number of devices and methods allowing the identification ofobjects by means of sensors are known as state-of-the-art. Thesensor-based identification of objects can be useful for a variety ofapplications. Parts produced in a mass production process can beautomatically examined for specific features, for example, in order todetect errors or defects. Sensor-based object identification is alsouseful in connection with manipulators such as industrial robots, forexample, to permit automatic gripping or handling of objects by means ofa manipulator. Safe sensor-based object identification is also importantin the control of autonomous driverless transport systems. In this case,object identification can be used for locating transport goods in alogistics scenario, for example, or to enable the targeted andautonomous control of objects.

An optical inspection system, with which manufactured objects (e.g.components) can be inspected for errors and defects, is previously knownfrom DE 10 2014 211 948. A camera, for example, which provides athree-dimensional point cloud of the objects to be identified, should beused for object identification. The point cloud should be a group ofthree-dimensional points on the surface of a solid object, and can alsobe derived from CAD volume models, for example. In order to be able toidentify objects in the environment, the three-dimensional point cloudsprovided by the camera are compared with previously known point cloudtemplates, so as to, for example, be able to identify previously storedobjects.

A method for the sensor-based identification of three-dimensionalobjects is also previously known from the publication “Model Globally,Match Locally: Efficient and Robust 3D Object Recognition” by BertramDrost, et al. A three-dimensional point cloud is generated in thispreviously known method as well, and in each case two points on thesurface of the point cloud are combined to form point pair features. Thepoint pair features are formed from specific geometric parameters of thetwo selected points, such as the distance between the two selectedpoints and the angle of the normal to the distance line between the twopoints. This process is relatively complex, however, because it requiresthree-dimensional image processing, among other things, and a largenumber of point pair features have to be considered. In addition, theultimate decision whether a detection hypothesis is accepted or rejectedis based solely on a voting and clustering method, which is consideredto be not robust enough for the application scenario of the presentinvention.

It is therefore an object of the present invention to provide animproved method for controlling a robot and/or an autonomous driverlesstransport system, with which a reliable and simple identification ofobjects is possible with minimum effort to enable a suitable control. Anew method for deciding the acceptance or rejection of a detectionhypothesis is presented as well. Taking into account physical andgeometric properties of the measurement process, said method checkswhether the detection hypothesis is consistent with the obtainedmeasurement data. Furthermore, the voting method presented here differsfrom the method in the aforementioned publication of Drost et al.

SUMMARY

The aforementioned object is achieved with a method for controlling arobot and/or an autonomous driverless transport system as described andclaimed herein.

The aforementioned object is in particular achieved by means of a methodfor controlling a robot and/or an autonomous driverless transport systemin which an improved sensor-based identification of objects is used. Asa first step, an object model of an object to be identified is providedin the form of a 2D surface contour. This can preferably be accomplishedby means of a manual or an automatic processing of a CAD model of theobject to be identified. The object model can, for example, be a set ofgeometric primitives, such as circles, polygons, etc., which have adefined inner and outer side. The object model thus advantageouslycorresponds to a reduction of the object to be identified onto arepresentation, which can be identified, for example by the sensor to beused later. In the case of a horizontally mounted 2D laser scanner, forexample, this can be a horizontal section through the object at thelevel of the laser scanner. In addition, an object reference pose to bedefined as desired is preferably provided for the object model of theobject, which can for example but not necessarily correspond to thecoordinate origin with orientation zero used in the CAD model.

The object model in the form of a 2D surface contour can be stored in adatabase, for example, which can be arranged remotely, for example, oris synchronized with a remote database, and which thus makes the dataavailable to a wide variety of users or applications. If the method isused in conjunction with autonomous driverless transport systems, forexample, the transport systems or the control devices of the transportsystems can access the database, which may be located in a computingcenter, for example wirelessly, and retrieve the required data. Multipletransport systems can thus access the same data, which, for example,considerably simplifies the updating of the data being used.

A model point cloud is generated from the object model, for example, byplacing points (model points) at configurable, regular intervals on the2D surface contour (or the 2D contours) of the object model. Each ofthese points is associated with a surface normal, which corresponds tothe surface normal of the 2D surface contour at this location. The modelpoint cloud corresponds to this set of oriented points generated in thisway.

As a further step, point pair features are formed on the basis of themodel point cloud (i.e. based on the 2D surface contour). To do this,for example, a subset of all possible point pairs of the model points isconsidered. This subset can be randomly or systematically selected, forexample. For each point pair of this subset, a point pair featuref=(d₁₂, α₁, α₂) is formed. The point pair features are based at least onthe distance d₁₂ between the points P₁ and P₂ of a point pair and thetwo angles α₁ and α₂ of the surface normal of each of the points to thedistance line between the two points. Depending on the complexity of theobject to be identified, several 1,000 or several 10,000 point pairfeatures of a 2D surface contour, for example, can be formed.

A feature pose is also preferably assigned on the basis of a point pairfeature as follows: a point M is defined, which lies on the center ofthe connecting line between the points of a point pair, for example. Inaddition, a direction vector between M and one of the points of thepoint pair is defined. The position component of the feature pose isobtained as the point M and the orientation component is obtained as theorientation of the direction vector. Furthermore, for the respectivepoint pair feature, the relative object pose (i.e. the object referencepose) of the object to be identified relative to the feature pose isdefined. The point pair features of the 2D surface contour of the objectto be identified (also referred to herein as “model features”) are thuslinked to a relative object pose.

The model features (including the relative object poses) couldoptionally also be precalculated and, like the 2D surface contour,stored in a database, which can be arranged remotely, for example, orcould be synchronized with a remote database.

As a further step, at least one 2D point cloud of the environment(environment point cloud) is acquired by means of a distance sensor, andcorresponding point pair features of the acquired 2D point cloud of theenvironment are formed, i.e. the point pair features are formed in thesame way as the model features. A surface normal is thus estimated foreach point analogously to the point pair features. The environment pointcloud thus consists of a set of oriented points, for example. Thedistance sensor is generally preferred to be a laser scanner, mostpreferably a 2D laser scanner. The point pair features of the 2D pointcloud of the environment formed in this way are compared with the pointpair features of the 2D surface contour (the model features) and similarpoint pair features are identified. On the basis of the point pairfeatures determined to be similar, it is then determined, preferably bymeans of a voting method, whether the object to be identified has beenidentified or not, and the robot and/or the autonomous driverlesstransport system is accordingly controlled in order to grab theidentified object, for example, or to move toward said object.

The step of comparing the point pair features preferably comprises adetermination of pose hypotheses of the object to be identified on thebasis of the point pair features determined to be similar. For example,on the basis of the environment point cloud, or on the point pairfeatures resulting therefrom, hypotheses about the pose of the object tobe identified are generated in the course of a voting method (so-calledpose hypotheses).

The voting method can preferably be carried out as follows: As a firststep, point pair features of the 2D point cloud of the environment aregenerated or determined, that all originate from exactly one point(reference point). To do this, a subset of the environment point cloudis selected, for example, and each point of this subset participates asa reference point in a voting method to generate pose hypotheses. Therespective reference point is preferably paired with other points of theenvironment point cloud and, for each such point pairing, a point pairfeature is calculated in the same way as the point pair features of themodel point cloud used for comparison. These point pair features arealso referred to in the following as environment features. In contrastto the model features, however, the environment features are notassociated with a relative object pose, because the determination of theobject pose is specifically the purpose of the voting method. For thispurpose, similar model features are sought for each environment featuregenerated in a voting method and for each similar model feature found atransformation is determined, which displays the feature pose of themodel feature on the feature pose of the environment feature. Therelative object poses associated with the model features are transformedinto this scene by means of the respective transformation, therebygenerating preliminary pose hypotheses.

The pose hypotheses are then preferably entered into a voting grid. Thevoting grid is in turn a partitioning of the pose hypothesis space intodiscrete cells, whereby the cells contain the (preliminary) posehypotheses that fall within them. The voting grid is preferablyefficiently implemented by means of a hash table, whereby the keycorresponds to the three-dimensional cell index, which in turncorresponds to a discretization of the pose hypotheses. Once all thepreliminary pose hypotheses, which in this step are all preferably basedon one and the same reference point, are entered into the voting grid,at least one preliminary pose hypothesis is selected from the votinggrid and defined as the resulting pose hypothesis of the voting method.Preferably only the cell with the most pose hypotheses is considered forthis purpose, and one pose hypothesis (is) preferably randomly selectedfrom the poses contained in this cell. The other pose hypotheses of thevoting grid are discarded and not considered. The voting grid issubsequently deleted.

The voting method by means of the voting grid is preferably repeatedindependently for several reference points, so that several resulting orprobable pose hypotheses are determined. However, the quantity ofresulting pose hypotheses found in this way is considerably smaller thanthe quantity of all the preliminary hypotheses ever entered into thevoting grids. Since the resulting or probable pose hypotheses selectedin this way have emerged as a result of consensus formation during thevoting method (e.g. in each case selected from the voting grid cell withmost poses), they are more probable hypotheses, in the sense that theactual object pose has a greater probability of being in the vicinity ofa resulting (probable) pose hypothesis than on a randomly selectedpreliminary pose hypothesis. As a result of this reduction of thequantity of pose hypotheses, the subsequent processing steps, with whichthe resulting pose hypotheses are preferably further optimized andsubjected to a consistency check, are accelerated.

In an advantageously subsequent optimization step, the pose hypothesesresulting from the respective voting methods are changed with the intentto increase the accuracy of the pose estimation even more. This ispreferably carried out by means of an Iterative Closest Point Method(ICP) in which, in each sub-step of the method, the points of theenvironment point cloud are associated with the respective closest pointof the points of the model point cloud transformed into the scene.Points are preferably associated only if the distance of the points doesnot exceed a given limit value, and the angular difference between thesurface normals of the points does not exceed a given limit value. Onthe basis of the points associated in this way and by means of asingular value decomposition, a new pose hypothesis, which minimizes thesum of the squared point distances of the respectively associatedpoints, can efficiently be found. This new thus determined posehypothesis is the starting point for the next iteration of the ICPmethod, which is carried out until convergence or until a given maximumnumber of iteration steps for the pose hypothesis has been reached.

In the course of determining whether the object to be identified hasbeen correctly identified or not, the resulting and optionally optimizedpose hypotheses are preferably subjected to a consistency check in orderto identify false hypotheses as such and discard them. The consistencycheck can be based on a variety of individual criteria, which can beused individually or also cumulatively. In the following, it is assumedthat the object model has been transformed into the pose of therespective pose hypothesis to be tested, and the points of theenvironment point cloud have been associated with the points of thetransformed 2D surface contour or the corresponding model point cloud.In doing so, an environment point is associated with the closest modelpoint of the 2D surface contour—but only if the distance of the pointsand the angular difference of the surface normals of the points do notexceed a particular limit value.

A first criterion is based on the free space model of the object, whichis interpreted as a region within which no objects other than the objectitself are expected to be. The object model of a table, for example,could consist of four circles that model a horizontal section throughthe table legs at the level of the distance sensor, and the free spacemodel could be a rectangle that encompasses these four circles. If anattempt is made to identify this four leg table and it is erroneouslyassumed to be at a pose at which there is actually a similar table, butwith six legs, the additional two legs would result in measurementsthat, on the one hand, lie within the free space model of the four legtable, but that, on the other hand, cannot be associated with modelpoints of the four leg table. The first criterion is therefore based ona limit value for the number of points of the environment point cloudthat are located within the free space model of the object and thatcould not be associated with model points. In other words, the sensoracquires points at positions at which, if the pose hypothesis iscorrect, there should be none. If this limit value is exceeded for agiven pose hypothesis, the pose hypothesis is discarded.

A further criterion is based on the free space model of the sensor,which is interpreted as a region within which no obstacles have beenmeasured. In the case of a 2D laser scanner, this could be a polygon,for example, which consists of the position of the laser sensor and theend points of the laser beams. If, for example, an attempt is made toidentify a six leg table consisting of six circles in the object model,and it is erroneously assumed to be at a pose at which there is actuallya similar table, but with four legs, the model points of two of the legsof the six leg table would lie within the free space model of the sensor(provided that the region is not hidden from the sensor by the objectitself). Model points would thus be present at points at which thesensor has not acquired anything. This additional criterion is thereforebased on a limit value for the number of points of the model point cloudthat are located within the free space model of the sensor. If thislimit value is exceeded, the pose hypothesis is discarded.

A further criterion is likewise based on the free space model of thesensor, but considers the maximum penetration depth of a model pointinto the free space model of the sensor (distance of the point to thecontour of the free space model, for points that lie within the freespace model). If this limit value exceeds a given limit value, the posehypothesis is discarded.

A further criterion is based on the ratio of the number of points of the2D surface contour (model points) of the object to be identified whichshould in fact be acquired by the distance sensor (expected modelpoints), to the actually acquired points of the 2D point cloud of theenvironment (observed model points). The expected model points caneasily be calculated geometrically, because the pose hypothesisspecifies the position of the object model (e.g. based on the CAD modeland the model point cloud) relative to the pose of the sensor and, also,the viewing areas of the sensors can be presumed to be known withsufficient accuracy. The expected model points thus include only themodel points theoretically visible from this perspective, which excludesthe points on the rear side of the model, for example, as well as pointsfrom regions that are not visible because they are concealed by theobject itself. In order to determine the observed model points, theexpected model points are now associated with points of the environmentpoint cloud (instead of, as has been the case previously, theenvironment points to the model points), whereby again maximum pointdistances and angle differences are taken into account. An expectedmodel point is then also an observed model point, if it was possible toassociate it with a point of the environment point cloud. The ratio ofthe number of observed model points to expected model points is one, ifall the expected model points have been observed, and goes to zero themore the object is covered, for example, by other objects, or the morethe actually observed object deviates from the expected object model atthis pose. This additional criterion therefore defines a limit value forthis ratio and the pose hypothesis is discarded, if this limit value isundershot.

A further criterion is based on the number of observed areas of theobject model. To do this, the object model is divided into differentregions by the user or each geometric primitive (e.g. circle, polygon,etc.) of the object model is implicitly interpreted as a separate area.When preparing the model point cloud from the object model, the regioneach model point was generated from is recorded. A region is consideredto have been observed if at least one model point linked to it is anobserved model point, i.e. when a point of the acquired 2D point cloudcan be assigned to the region. This criterion defines a limit value forthe number of observed areas and the pose hypothesis is discarded, ifthe limit value is undershot.

If a pose hypothesis has not been discarded, the object is considered tobe acquired at this point. An accepted pose hypothesis then discardssimilar pose hypotheses which are located within a translational androtational tolerance range around the accepted pose hypothesis. The thusdiscarded pose hypotheses do not have to be processed further, whichaccelerates the subsequent processing steps.

The acquisition method does not return a pose if the object has not beenacquired (e.g. because it really does not exist in the scene), and themethod can return multiple poses if the object exists more than once inthe scene. The same physical object can also be acquired at a number ofposes if the object exhibits symmetries. A 90 degree rotationallysymmetrical object, for example, is found at four different poses withideally identical position components but different orientationcomponents.

After the identification of an object or the determination of a posehypothesis as being correct, the object is preferably tracked using SMCfilters, in order to increase the accuracy of the determination of therelative arrangement of the sensor and the object, even in the event ofa movement of the two elements relative to one another. SMC filters arealso known as particle filters, and allow the exact and continuouslyupdated determination of the location and the velocity of an object.Specifics regarding these methods can, for example, be found in thereference book “Probabilistic Robotics”, The MIT Press, 2005.

The invention further relates to an autonomous driverless transportsystem comprising a system for the identification of objects, whichcomprises at least one distance sensor and a control device that isconfigured to execute the above-described methods. The steps to beexecuted by the control device in particular comprise the retrieval ofpoint pair features of at least one 2D surface contour of an object tobe identified from a database, wherein the point pair features are basedat least on the distance between the two points of a point pair, and thetwo angles of the normal (surface normal or curve normal) of each of thepoints to the distance line between the two points. The database can beintegrated in the control device; however, it can also be an externaldatabase that can be accessed by the control device. It is indeedpreferable for the database with the 2D surface contours of the CADmodels to be a central database, or to be synchronized with a centraldatabase that can be used by multiple systems. This simplifies theupdating or the addition of data from CAD models.

The control device is furthermore configured to acquire at least one 2Dpoint cloud of the environment by means of the distance sensor, and toform corresponding point pair features of the acquired 2D point cloud ofthe environment. The control device is further configured to comparepoint pair features of the 2D surface contour with the point pairfeatures of the 2D point cloud of the environment and to determinesimilar point pair features. Such a comparison can also be performed ona separate computer, for example, that is accessed by the controldevice. The control device is further configured to be able to determinesimilar model point pair features and, based on these, preferably in avoting method, to determine pose hypotheses. These can then preferablybe optimized in a further step and, after a consistency check, either beaccepted or rejected as an acquisition.

The method steps that are executed by the control device substantiallycorrespond to the method steps described above, so that there is no needfor a renewed detailed explanation here.

BRIEF DESCRIPTION OF THE FIGURES

The present invention is explained by way of example with reference tothe attached figures, in which:

FIG. 1 shows a schematic side view of an autonomous driverless transportsystem that can execute a method according to the invention;

FIG. 2 depicts a plan view onto the arrangement of FIG. 1;

FIG. 3 illustrates an example of the determination of point pairfeatures based on a 2D surface contour;

FIG. 4 illustrates the formation of point pair features of the generated2D surface contour of a CAD model;

FIG. 5 illustrates the formation of corresponding point pair features ofthe acquired 2D surface contour;

FIG. 6 schematically illustrates the procedure for the consistency checkon the basis of the free space model of the object;

FIG. 7 schematically illustrates the procedure for the consistency checkon the basis of the free space model of the sensor;

FIG. 8 schematically illustrates the procedure for the consistency checkon the basis of the free space model of the sensor and the penetrationdepth of a model point;

FIG. 9 schematically illustrates the procedure for the consistency checkon the basis of a comparison of expected and actually observed modelpoints; and

FIG. 10 schematically illustrates the procedure for the consistencycheck on the basis of the observed regions of the object model.

DETAILED DESCRIPTION

FIG. 1 shows a schematic side view of an autonomous driverless transportsystem 10, which has a distance sensor in the form of a 2D laser scanner20 that is connected to a control device 15. The control device 15 islocated on the body 12 of the transport system 10, which can be moved bymeans of wheels 13. In the vicinity of the transport system 10 there isa table 30 which has a number of legs 31, 32, 33, 34. The intent is forthe transport system 10, for example, to approach a specific positionrelative to the table 30. For this purpose, the transport system 10requires information that would enable the transport system 10 toperform an autonomous alignment with respect to the table 30. Thisinformation can be provided by the method according to the invention.

FIG. 2 shows a plan view of the arrangement of FIG. 1. The tabletop 30of the table is not shown for illustrative purposes, but four legs 31,32, 33 and 34 can be seen. The 2D laser scanner 20 of the transportsystem 10 emits a fan of laser beams 21, which are indicated in FIG. 2with dashed arrows 21. When the laser beams 21 strike an object, such asthe legs of the table, the beams are reflected, thus allowing the 2Dlaser scanner 20, or the associated control device 15, to generate atwo-dimensional image of the environment, i.e. a 2D point cloud of theenvironment. This is explained in more detail below with reference toFIGS. 4 and 5.

As an example, the formation of point pair features of a 2D surfacecontour K will be explained with reference to FIG. 3. The 2D surfacecontour K corresponds to a two-dimensional curve and can, for example,be generated from a three-dimensional CAD model by placing atwo-dimensional section through said CAD model. Two arbitrary points P₁and P₂ are now selected from this 2D surface contour, and the normals n₁and n₂ of these two points are formed. In addition, the distance and thedistance line d₁₂ between the two points P₁ and P₂ is determined. Withthis data, it is then possible to determine the two angles α₁ and α₂ ofthe normals n₁ and n₂ of the points P₁ and P₂ to the distance line d₁₂.These values of the point pairs P₁, P₂ together form a point pairfeature, i.e. f=(d₁₂, α₁, α₂). A feature pose is preferably defined aswell, in which a point M is defined that lies, for example, on themiddle of the connecting line d₁₂ between the points of the point pair.In addition, a direction vector between M and one of the points of thepoint pair is defined, as shown by the continuous arrow originating fromM in FIGS. 3 and 4. The position component of the feature pose istherefore the point M and the orientation component is the orientationof the direction vector. This allows a unique assignment of each pointpair feature to the object pose of the object to be identified. Asindicated in FIG. 3, for the point pair feature determined or defined inthis way, the object pose (object reference pose) of the object to beidentified can be defined or assigned relative to the feature pose. Inthe two-dimensional space, the x- and the y-distance of the point M fromthe position of the object pose and the angle between the two directionvectors of the feature pose and the object pose are sufficient for thispurpose. Each point pair feature of the 2D surface contour of the objectto be identified is thus linked to the respective relative object pose.

This process is repeated for a large number of point pairs on the 2Dcontour K, for example for several 100 or several 1,000 point pairs. Inthis way, a plurality of point pair features of the 2D surface contour Kare formed, which are ultimately representative of the two-dimensionalshape of the surface contour K. In addition, each point pair feature isassigned the respective object pose. If the contour K of the 2D surfacecontour corresponds to a section through a three-dimensional CAD model,for example, the point pair features determined in this way form amathematical description of the object to be identified, which can bestored in a database.

In the course of acquiring a 2D point cloud of the environment by meansof the 2D laser scanner 20, for example, a plurality of points of theenvironment are acquired, depending on how many reflective surfacesthere are within the range of the distance sensor. Corresponding pointpair features are formed for this acquired 2D point cloud of theenvironment as well, i.e. analogous to the point pair features of theobject to be identified, which can then be compared to the point pairfeatures of the object to be identified stored in the database.

In FIG. 4, the dotted rectangles 31′, 32′, 33′, 34′ should have beengenerated from a CAD model of the table 30, whereby the CAD model of thetable 30 was cut two-dimensionally at the level of the scanning plane ofthe 2D laser scanner 20. For this purpose, an object model of the objectto be identified in the form of a 2D surface contour was generated fromthe CAD data, for example, onto which points were then placed atconfigurable intervals. A comparison with FIG. 2 shows that the dottedcontours 31′ to 34′ correspond to the contours of the legs 31 to 34. Thepoints ultimately permit a comparison with the actual sensor data.

For this purpose, as indicated in FIG. 4 and explained above withreference to FIG. 3, a plurality of point pair features of the contours31′ to 34′ are formed on the basis of the 2D surface contour of thetable legs. The surface normals n₁, n₂, as well as the distance d₁₂between the two points P₁ and P₂ of the respective point pair, aredetermined for all the points of the pairs. The angles of the surfacenormals of each point to the connecting line d₁₂ between the two pointsP₁ and P₂ are then determined, as well as a point M with the associateddirection vector. In this example therefore, a point pair featureconsists of at least of three values: the distance between the points P₁and P₂ (d₁₂), and the angles α₁ and α₂ of the surface normal of eachpoint to the connecting line between the points. In addition, each pointpair feature of the 2D surface contour is linked to the respectiverelative object pose by means of the point M and the direction vector.The data obtained in this way is stored.

FIG. 5 schematically shows how the 2D surface contours of the real legs31 to 34 of the table 30 are acquired by the 2D laser scanner 20 in theform of 2D point clouds. It is obvious to the person skilled in the artthat the depiction in FIG. 5 represents a simplification, because, inreality, the 2D laser scanner 20 will acquire many other surfacecontours of the environment, such as the walls in a factory hall, forexample, or other objects that are not part of the table 30. Theacquired 2D point clouds 31″, 32″, 33″ and 34″ are not identical to the2D surface contours or model point clouds 31′ to 34′ derived from theCAD model, because the visual range of the 2D laser scanner is limitedand cannot detect the rear sides of the legs 31 to 34, for example.Nonetheless, the similarity of the acquired 2D point clouds 31″ to 34″to the 2D surface contours or model point clouds 31′ to 34′ determinedfrom the CAD model is sufficient. This ultimately makes it possible todetermine whether the object to be identified, i.e. the “table 30”, hasbeen identified or not. Corresponding point pair features are formedfrom the acquired 2D point clouds 31″ to 34″ (in order to distinguishthe values necessary for the formation of the point pair features fromthose of FIG. 4, the values are labeled with i and j in the subscript inFIG. 5). A plurality of point pair features of the acquired 2D pointclouds is thus obtained.

In a subsequent step, to determine similar point pair features, thepoint pair features of the model are compared with the point pairfeatures of the acquired 2D point clouds. Since, in practice, the 2Dlaser scanner 20 has acquired more contours than those shown in FIG. 5,the point pair features of the acquired 2D point clouds of theenvironment include a large number of point pair features that differgreatly from the point pair features of the model, because the distanced_(ij), for example, is many times greater than the largest possibledistance d₁₂ from the CAD model.

In the present example, for the determination of similar point pairfeatures, use can also be made of the fact that the distance d₁₂ alwayshas to be within certain ranges. This is because the two points P₁, P₂of a pair are either located on the same contour, e.g. both on thecontour 31′, or they are located on or in different contours. If theyare located in different contours, the distance d₁₂ has to approximatelycorrespond to (1.) the distance between the two contours 31′ and 32′, or(2.) the distance between the contours 31′ and 33′, or (3.) the distancebetween the contours 31′ and 34′. In the example shown, the distance d₁₂of all point pair features of the 2D surface contour of the model musttherefore lie within one of four exactly definable ranges. Point pairfeatures of the acquired 2D point clouds of the environment, in whichthe distance d_(ij) is far from these four ranges can be rejectedimmediately as non-similar.

FIGS. 6 to 10 schematically illustrate the procedure for the consistencycheck, i.e. after the (probable) pose hypotheses resulting from theabove-described methods, for example, have been determined. Theconsistency check is intended to make the procedure for determiningwhether the object to be identified has been correctly identified or notmore precise and more reliable. In the consistency check, previouslydetermined pose hypotheses are tested so as to be able to detect andreject false hypotheses. The consistency checks explained in more detailbelow can respectively be used alone or in combination. In thefollowing, it is assumed that the object model has been transformed tothe pose of the respective pose hypothesis currently to be tested (orvice versa), and the points of the environment point cloud have beenassociated with the points of the transformed 2D surface contour, or thecorresponding model point cloud.

FIG. 6 schematically illustrates the procedure for the consistency checkbased on the free space model of the object. In the situation shown, onthe basis of the acquired laser points 61″, 62″, 63″, 64″, a table withfour legs is suspected to be at a specific position (hypothesis). Themodel point clouds 61′, 62′, 63′, 64′ have been generated as in theexample of FIG. 4, for example from a CAD model of the table. In realitythere is a table with six legs at that position, so that additionalpoints are acquired at the position indicated by the reference sign 66″.The object model of the suspected 4 leg table consists of the four modelpoint clouds 61′, 62′, 63′, 64′ of the table legs and, in the exampleshown, the free space model of the object is a rectangle 65 thatencompasses the four model point clouds 61′, 62′, 63′, 64′. No otherpoints may be acquired by the sensor within the free space model 65, ora limit value is defined for the number of points of the environmentcloud that are within the free space model 65, but cannot be assigned toone of the four model point clouds 61′, 62′, 63′, 64′. The additionallyacquired point cloud 66″ is therefore acquired at a position in the freespace model 65, at which there should actually not be any points to beacquired. The pose hypothesis, that there is a table with four legs atthe position scanned by the sensor 20, is therefore discarded as false.

FIG. 7 schematically shows the procedure for the consistency check basedon the free space model of the sensor. In this situation, a table withsix legs is suspected to be at a position (pose hypothesis) at whichthere is actually a table with four legs. The pose hypothesis istherefore false. The free space region 77 of the sensor is a region inwhich no environment points have been acquired/measured. The six modelpoint clouds 71′, 72′, 73′, 74′, 75′, 76′ are in turn generated from aCAD model of the 6 leg table, for example. Based on the acquired laserpoints 71″, 72″, 75″, 76″, a table with six legs is thus suspected to beat a specific position; in the free space model 77 of the sensor,however, there are model point clouds 73′ and 74′ that cannot beassociated with a laser point. Therefore, there are model points ormodel point clouds at positions in which the sensor has not acquiredanything. If the number of these model points exceeds a specific limitvalue, the pose hypothesis is discarded.

FIG. 8 schematically shows the procedure for the consistency check basedon the free space model of the sensor and the penetration depth of amodel point. In the situation shown, a 4 leg table is suspected at aposition that deviates from the actual position or pose of an actuallypresent 4 leg table. The actual position is shown by means of theacquired point clouds 81″, 82″, 83″, 84″ in FIG. 8. The pose hypothesisto be tested, however, suspects the table to be at the position shown bymeans of the model point clouds 81′, 82′, 83′ and 84′ in FIG. 8.Similarly to the procedure for the example of FIG. 7, limit values formodel points are considered here, which lie within the free space model88 of the sensor. In doing so, however, the distance 87 betweenindividual model points within the free space model 88 and theboundaries of the free space model 88 is important. If the distance 87exceeds a specific limit value, the pose hypothesis is discarded. If,however, the model points are located close to the boundary of the freespace model 88, for example, the pose hypothesis can under certaincircumstances be considered to be correct.

FIG. 9 schematically shows the procedure for the consistency check basedon a comparison of expected and actually observed model points. Thisapproach is based on the ratio of the number of points of the 2D surfacecontour of the object to be identified that should in fact be acquiredby the distance sensor (expected model points), to the actually acquiredpoints of the 2D point cloud of the environment (observed model points).In the example shown, a 6 leg table is suspected to be at a position(pose hypothesis) at which there is actually a 6 leg table. The posehypothesis is therefore correct. However, two legs of the table areconcealed by an obstacle 90. Model points 91 a′, 92 a′, 93 a′, 94 a′, 95a′ and 96 a′ can be determined from the geometric calculation, that arenot expected to be able to be acquired by the sensor, because, from theview of the sensor 20, they are located on the rear side of a table leg(such as 91 a′, 92 a′, 93 a′, 95 a′ and 96 a′) for example, or becausethey are concealed by another table leg (such as 94 a′). A number ofmodel points 91′, 92′, 93′, 95′ and 96′ are additionally determined,which should actually be acquired by the sensor. In the case of 93″, 95″and 96″, the sensor does indeed acquire points at positions in which itis expected. At positions 91′ and 92′, however, the sensor does notacquire any points due to the obstacle 90, even though it is expected.The consistency check is then based, for example, on the ratio of theexpected and the actually observed model points 93″, 95″ and 96″ to thetotal number of expected points.

FIG. 10 schematically shows the procedure for the consistency checkbased on the observed areas of the object model. In the example shown, a6 leg table is again suspected to be at a position (pose hypothesis) atwhich there is actually a 6 leg table. The pose hypothesis is thereforecorrect. However, two table legs are concealed by an obstacle 100, sothat the sensor 20 cannot acquire them, and a further leg is notacquired because it is concealed by another leg. Points are thereforeacquired only at the positions 103″, 105″ and 106″. In the exampleshown, the object model is divided by the user into six rectangularregions 101, 102, 103, 104, 105, 106, and each of these rectangles ofthe object model is considered as a separate region. When preparing themodel point cloud from the object model, the region each model point wasgenerated from is recorded. A range is considered to have been observedif at least one model point linked to it is an observed model point,i.e. if a point of the acquired 2D point cloud can be assigned to theregion. In the example of FIG. 10, the regions 103, 105 and 106 havebeen acquired, namely by the point clouds 103″, 105″ and 106″, whereasthe regions 101, 102 and 104 have not been acquired. The pose hypothesiscan be discarded, for example, if a specific limit value for the numberof observed regions is undershot.

While the present invention has been illustrated by a description ofvarious embodiments, and while these embodiments have been described inconsiderable detail, it is not intended to restrict or in any way limitthe scope of the appended claims to such detail. The various featuresshown and described herein may be used alone or in any combination.Additional advantages and modifications will readily appear to thoseskilled in the art. The invention in its broader aspects is thereforenot limited to the specific details, representative apparatus andmethod, and illustrative example shown and described. Accordingly,departures may be made from such details without departing from thespirit and scope of the general inventive concept.

LIST OF REFERENCES

10 Autonomous driverless transport system 12 Body 13 Wheels 15 Controldevice 20 Distance sensor/laser scanner 21 Laser beams 30 Table 31, 32,33, 34 Legs 31′, 32′, 33′, 34′ 2D surface contour or point cloud fromCAD model 31″, 32″, 33″, 34″ Sensor-acquired 2D point clouds 61′, 62′,63′, 64′ 2D surface contour or point cloud from CAD model 61″, 62″, 63″,64″, 66″ Sensor-acquired 2D point clouds 65 Free space model of theobject 71′, 72′, 73′, 74′, 75′, 76′ 2D surface contour or point cloudfrom CAD model 71″, 72″, 75″, 76″ Sensor-acquired 2D point clouds of thelegs 77 Free space model of the sensor 81′, 82′, 83′, 84′ 2D surfacecontour or point cloud of the legs from CAD model 81″, 82″, 83″, 84″Sensor-acquired 2D point clouds 88 Free space model of the sensor 87Distance boundary free space model and model point 91′, 92′, 93′, 95′,96′ Model points expected to be able to be acquired by the sensor 91a′,92a′, 93a′ . . . 96a′ Model points not expected to be able to beacquired by the sensor 93″, 95″, 96″ Expected and acquired model points90, 100 Obstacle 101, 102, 103 . . . 106 Regions of the object model103″, 105″, 106″ Sensor-acquired 2D point clouds

What is claimed is:
 1. Method for controlling a robot and/or anautonomous driverless transport system (10) comprising the followingsteps: Provision of an object model of an object to be identified in theform of a 2D surface contour; Formation of point pair features based onthe 2D surface contour, wherein the point pair features are based atleast on the distance (d₁₂) between the two points (P₁, P₂) of a pointpair, and the two angles (α₁, α₂) of the normal (n₁, n₂) of each of thepoints (P₁, P₂) to the distance line between the two points (P₁, P₂);Detection of at least one 2D point cloud of the environment by means ofa distance sensor (20), and formation of corresponding point pairfeatures of the acquired 2D point cloud of the environment; Comparisonof the point pair features of the 2D surface contour with the point pairfeatures of the 2D point cloud of the environment and determination ofsimilar point pair features; and Based on the point pair featuresdetermined to be similar: determination whether the object to beidentified (31, 32, 33, 34) has been identified or not, and Control ofthe robot and/or autonomous driverless transport system (10). 2-17.(canceled)